Finite-Sample Analysis for Decentralized Batch Multiagent Reinforcement Learning With Networked Agents

Despite the increasing interest in multiagent reinforcement learning (MARL) multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this article, we address problem by providing finite-sample analysis for decentralized batch MARL. Specifically, consider type of mixed MARL setting with both cooperative and competitive agents, where two teams agents compete zero-sum game setting, while within each team collaborate communicating over time-varying network. This covers many conventional settings literature. We then develop algorithms that can be implemented fashion, quantify errors estimated action-value functions. Our error captures how function class, number samples iteration, iterations determine statistical accuracy proposed algorithms. results, compared to bounds single-agent learning, involve additional terms caused computation, which is inherent our setting. article provides first MARL, step toward rigorous general regime.